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ABSTRACT 

In recent decades, NASA’s interest in spacecraft rendezvous and proximity operations has grown. Additional 
instrumentation is needed to improve manned docking operations' safety, as well as to enable telerobotic operation of 
spacecraft or completely autonomous rendezvous and docking. To address this need. Advanced Optical Systems, Inc., 
Orbital Sciences Corporation, and Marshall Space Flight Center have developed the Advanced Video Guidance Sensor 
(AVGS) under the auspices of the Demonstration of Autonomous Rendezvous Technology (DART) program. Given a 
cooperative target comprising several retro-reflectors, AVGS provides six-degree-of-freedom information at ranges of 
up to 300 meters for the DART target. It does so by imaging the target, then performing pattern recognition on the 
resulting image. Longer range operation is possible through different target geometries. 

Now that AVGS is being readied for its test flight in 2004, the question is: what next? Modifications can be made to 
AVGS, including different pattern recognition algorithms and changes to the retro-reflector targets, to make it more 
robust and accurate. AVGS could be coupled with other space-qualified sensors, such as a laser range-and-bearing 
finder, that would operate at longer ranges. Different target configurations, including the use of active targets, could 
result in significant miniaturization over the current AVGS package. We will discuss these and other possibilities for a 
next-generation docking sensor or sensor suite that involve AVGS. 

Keywords: Advanced Video Guidance Sensor (AVGS), 6 DOF information, laser range-and-bearing finder, 
autonomous rendezvous and docking. Demonstration of Autonomous Rendezvous Technology, space sensors, pattern 
recognition 


1. INTRODUCTION 

Over the last few decades, the purpose of our missions to space has broadened. Where once our nation’s space program 
was focused primarily on launching people into orbit and beyond, it now must maintain satellites, space-based 
telescopes, and other objects in orbit. Due in part to these new requirements, NASA has become more and more 
interested in spacecraft rendezvous and proximity operations. While NASA has performed many rendezvous in the past, 
each has been performed with a superbly trained astronaut at the controls, with little in the way of support 
instrumentation. Additional instrumentation is needed to improve the safety of manned docking operations as well as to 
enable telerobotic operation of spacecraft or completely autonomous rendezvous and docking. 

To address this need, Advanced Optical Systems, Inc. (AOS), Orbital Sciences Corporation (OSC), and Marshall Space 
Flight Center (MSFC) have developed the Advanced Video Guidance Sensor (AVGS) under the auspices of the 
Demonstration of Autonomous Rendezvous Technology (DART) project 1 . AVGS builds on the heritage of the MSFC 
Video Guidance sensor 2 . Given a cooperative target comprising several retro-reflectors, AVGS provides six-degree-of- 
freedom information at ranges of up to 300 meters for the DART target. It does so by imaging the target, then 
performing pattern recognition on the resulting image. Longer range operation is possible through different target 
geometries. Figure 1 shows an AVGS prototype unit. 
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Figure 1 . An AVGS prototype unit. 


Now that AVGS is being readied for its test flight in the latter half of 2004, the question is: what next? Modifications 
can be made to AVGS. including different pattern recognition algorithms and changes to the retro-reflector targets, to 
make it more robust and accurate. AVGS could be coupled with other space-qualified sensors, such as a laser range- 
and-bearing finder, that w'ould operate at longer ranges. Different target configurations, including the use of active 
targets, could result in significant miniaturization over the current AVGS package. In this paper, we discuss these and 
other possibilities for a next-generation AVGS or for a sensor suite that involves AVGS. Section 2 describes current 
AVGS operation in more detail. Section 3 discusses changes to the AVGS spot recognition algorithm. Section 4 covers 
possible changes to the targets and to AVGS that do not involve the spot recognition algorithm, while Section 5 
describes additional sensors that could be coupled with AVGS to increase overall performance. 

2. CURRENT AVGS OPERATION 

AVGS is a monostatic laser-based sensor that illuminates a target made of retro-reflectors, images the reflected light, 
and performs pattern matching on the resulting image 3 . A sample target is shown in Figure 2. AVGS employs two 
wavelengths of light, one at 850 nm, the other at 808 nm. The former is used to generate a foreground image; the latter, 
a background image. The retro-reflector targets include a piece of filter glass that passes 850 nm light and blocks 808 
nm, leading to a foreground image with bright spots from the retro-reflectors and a background image that lacks those 
spots. When the background image is subtracted from the foreground one, extraneous reflections subtract out, leaving 
only the spots corresponding to the retro-reflectors. 




Figure 2. A sample comer cube target used with AVGS. 


The returned light is collected by a telecentric lens and focused onto a CMOS-based imager. The image is then scanned 
for spots of the appropriate size and those spots' centroids calculated. To maintain a reasonable update rate, AVGS 
cannot operate on the entire subtracted image. Instead, the AVGS hardware performs the following algorithm. First, a 
threshold is applied, converting the grayscale image into a binary one of lit and unlit pixels. Lit pixels that are next to 
each other in a given row are then aggregated into line segments. Line segments that have at least one pixel’s worth of 
overlap are aggregated into spots, and each spot’s centroid is calculated. The software then searches for a group of spots 
of the appropriate size and whose configuration matches the known target configuration. 

Converting from a grayscale image to line segments and from line segments to spots involves a reduction in data. A 
given line segment consists of a starting and ending x coordinate, a y coordinate, the sum of the pixel’s intensities from 
the grayscale image, and the weighted sum of the intensities times their x coordinate. The latter two pieces of 
information are for the later centroid calculation. Spots contain less data than the line segments that make up the spot, 
and information about the pixels' distribution is lost. A given spot is defined by a bounding box that encompasses all of 
the pixels, the number of pixels in the spot, the sum of the pixels' intensities, and weighted sums of the intensities 
suitable for calculating the spot's centroid. Figure 3 gives a conceptual overview of this process, and shows how 
information about the actual pixels’ shape is lost. 
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Figure 3. A conceptual diagram of the transition from grayscale image to line segments to spots. 


The current AVGS design has much to recommend it. The spot-recognition algorithm, though simple, has the benefit of 
being remarkably robust. The distribution of light returned from the retro-reflector targets can be noticeably non- 
circular due to target damage or optical anomalies yet still produce valid spots under this algorithm. Its simplicity also 
makes implementation easier and limits the amount of processing time and memory required. Similarly, the current 
targets require no power and can withstand impacts that might damage more fragile or powered targets. 


There are some pitfalls to the design. If two spots ever have contiguous pixels, they are subsumed into a single spot. 
This has occurred during testing for the DART mission, in which some of the retro-reflector targets have lenses that 
spread their returned light and cause the spots to overlap, as shown in Figure 4 4 . Where a human can look at the 
original image and see two spots, the current spot algorithm cannot. The centroid calculation’s accuracy can be lessened 
by holes in the returned spots, as also seen in Figure 4. Spots are valid if their size is close to what is expected, but 
know ing what size to expect requires a priori knowledge of range. There are no automatic adjustments of laser power, 
imager integration time, or imager gain setting. Instead, laser power and integration time are set beforehand, and cannot 
be adaptively adjusted to accommodate unexpected changes in operating conditions such as underpowered lasers or 
extremely damaged targets. 



Figure 4. An image of targets that will result in two targets being treated as one spot (the upper two). 



3. AVGS SPOT RECOGNITION ALGORITHM CHANGES 

One obvious point of improvement is the algorithm that converts images to spots and then attempts to recognize them. 
A notable weakness of the current algorithm is its inability to deal with ov erlapping return spots. What is less obvious is 
what algorithm, if any, should replace or augment the current one. 

There are several possible ways to mitigate the current algorithm's inability to detangle overlapping spots. One such 
approach would be to use erosion or opening. Erosion removes pixels from the boundaries of objects, while dilation 
adds them. Opening consists of an erosion followed by a dilation, and is used to remove small objects from an image. 
Figure 5 shows an example of erosion on the image from Figure 4. 




Figure 5. Example of erosion. Image a) is eroded to give image b), then further eroded to give image c). 

Though erosion or opening could be used to separate the return spots, it might not be practical. Spots would have to be 
eroded noticeably before they separated, and any holes in the spots will be increased, possibly to the point that they 
would have an extremely negative effect on the centroid calculation. In addition, the erosion should ideally be 
performed on a full image, rather than on line segment data, since removing pixels in the vertical direction from 
horizontal line segments is difficult. Working with full images rather than line segments or spot data would require 
significant changes to the AVGS hardware. 

Another approach to distinguishing overlapping spots would be to create histograms of either the intensity or the 
number of lit pixels along x and y could be formed, as shown in Figure 6. The histograms could then be examined to 
determine if the spots actually contain two spots instead of just one. This could be accomplished by performing a curve 
fit, by looking for local maxima, by looking for inflection points, or by thresholding the histogram and seeing if doing 
so results in two separate peaks. The greatest drawback to this algorithm is that it introduces degenerate cases, as shown 
in Figure 7. The degeneracies would have to be broken through use of an additional, non-orthogonal axis, thus 







introducing more complexity: pixel images can easily be separated into x and y axes, while a non-orthogonal axis would 
require much more processing. 



Figure 6. Two spots and their histogram in x and y . 
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Figure 7. The histograms for the two sets of spots in a) and b) w ill be the same, 
even though they have a different overall configuration. 

Instead of trying to patch the current algorithm, it could be replaced entirely. One possible replacement algorithm is the 
circular Hough transform. The Hough transform is a method for finding arbitrary shapes in images by finding local 
patterns in parameter space \ One specific application of the Hough transform is in finding circles, even incomplete 
ones, of a specified radius 6 . Each edge point in an image is converted to a circle in parameter space. The parameter- 
space circle is of the same radius as the sought-after circles in image space. The parameter-space circles represent 
possible locations of the circle center. The most probable circle location is where the circles in parameter space overlap 
the most. Figure 8 shows four edge points and the resulting representation in parameter space. 



Figure 8. Circular Hough transform. The edge points in a) correspond to the dashed circles in b) in parameter space. 

To implement the circular Hough transform, first the image’s edge points must be identified. Then the edge points are 
turned into parameter-space circles and the circle locations collected in a two-dimensional accumulation array that 
corresponds to the image size. The accumulation array is then scanned for values above a set threshold. Every point 
above that threshold corresponds to the center of a circle. 

The circular Hough transform holds promise for AVGS target recognition. The retro-reflector targets produce returns 
that are circular in nature. The circular Hough transform would not be fooled by overlapping spots. In some ways, the 
circular Hough transform is also well suited to the current AVGS data model. Line segments are defined by start and 
end points, which are effectively the spot edge points. No extra edge locations would be required. However, there are a 
number of drawbacks to the method. You must specif' the circle size, and the transform can only find circles (or 
ellipses ox a ueuiieu ratio), "i ou must have an accumulation array whose size is equal to the image size to be searched. 
Some of these drawbacks can be mitigated: you can find a range of circle sizes if you do not care about finding the exact 
circle size, and if the current method of spot formation were used, the accumulation array size could be limited to the 
size of the spot's bounding box. 

4. AVGS AND TARGET CHANGES 

While changes to the spot recognition algorithm can increase AVGS performance, other gains are possible through 
hardware changes or changes to the AVGS targets. One possible hardware change would be replacing the current lens 
with one that is capable of zooming. The AVGS lens is telecentric and has a fixed focal length. It was not designed to be 
an imaging lens; rather, it was designed to produce a minimum spot size for an object at infinity that corresponded to a 
two-by-two pixel block. This restriction was necessary to allow AVGS software to distinguish return spots from single 
lit pixels due to defects in the CMOS imager. The resulting lens is extremely complex, requiring many pieces of glass to 
achieve the required telecentricity, minimum spot size, and distortion within the wide field of view. 

If that lens were replaced by one that could zoom and adjust its focus, then some of the lens complexity could be 
removed. The zoom amount could be chosen to give large enough spot returns at the maximum working distance. The 
spot returns’ holes would vanish, as they are an artifact of the lens not being a true imaging system. That in turn would 
improve the accuracy of the spot recognition algorithm's centroid calculation. For a zoom lens to be effective, the 
AVGS would have to scan through the possible ranges until it acquired a target, after which it could adjust the zoom 
adaptively as long as it was tracking the target. 

The entire acquisition and tracking process could be made adaptive. There are three settings that control how bright the 
imaged return spots are: laser power, imager integration time, and imager gain setting. Currently the imager gain setting 
is fixed, and the laser power and integration time are set beforehand using a third-order polynomial fit to laboratory 
experimental data. There are numerous drawbacks to this approach. The pattern of output laser light is Gaussian across 
the field of view, resulting in dimmer spots near the field of view edges than in the middle. To compensate, the laser 
power and integration time must be set to values that result in very oversaturated images in the center of the field of 
view. If the on-orbit targets do not reflect as much light as the ones used in the laboratory, then they may not be visible 
during actual AVGS operation. Instead, the laser power and imager integration time could be adaptively adjusted using 





a feedback loop. As with the zoom lens, the AVGS would scan through laser power and integration times until it 
acquired a target, after which it would adjust the settings to keep the return spots from being overly saturated. 

Several of the algorithms listed in Section 3 would require more data than the line segment and spot data that AVGS 
provides, in some cases requiring full image data. To implement those algorithms while maintaining the AVGS update 
rate of 25 Hz, either AVGS will have to be modified to make full image data available over a higher-speed link than the 
current serial bus, or more processing power would need to be added in order to perform the more computationally- 
intensive algorithms given in Section 3. 

All of the changes mentioned so far have been changes to AVGS units. However, AVGS units require targets in order to 
function properly. Those targets can be altered to provide different AVGS capabilities. One of the simplest changes 
would be replacing passive retro-reflector targets with active ones such as LEDs. This would have several benefits. For 
one, active targets would obviate the need for lasers on AVGS. This would in turn reduce power requirements, relax the 
current thermal requirements that are necessary to keep the lasers’ wavelengths stable, and potentially allow the overall 
AVGS unit to be miniaturized. For another, active targets might be of more interest to DoD organizations. Retro- 
reflectors are a poor addition to defense satellites, as it is generally a bad idea to attach shiny reflective things to 
satellites that should not be found by potentially hostile spacecraft. Active targets could be activated only upon 
receiving the proper code from a friendly AVGS unit. At all other times the targets could be inactive. 

5. ADDITIONAL SENSORS 

AVGS is well-suited for close operations, as it provides the necessary six-degree-of-freedom information from ranges 
of hundreds of meters into docking. It is not, however, a complete autonomous rendezvous and docking solution. The 
working range of AVGS depends on how far apart the retro-reflectors that compose a target are placed. The further 
apart the retro-reflectors are, the longer the distance at which AVGS will operate. For long-distance operation with 
small satellites, retro-reflectors would have to be placed on long boom arms. Range measurement accuracy decreases 
dramatically with increasing range, as shown in Figure 9. Because of this, an AVGS must be coupled with other longer- 
range sensors to form a complete solution. 
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Figure 9. AVGS range accuracy as a function of range for two targets used in the DART mission. 

Possible other space-based sensors include radar sensors, such as the Shuttle’s Ku-band system or the Kurs radar-and- 
transponder system aboard the Russian Progress spacecraft, and relative GPS systems. However, laser-based sensors, 
especially laser range-and-bearing finders, are currently underutilized. 


Laser range-and-bearing finders are conceptually similar to microwave radar, in that they broadcast a pulse of 
electromagnetic radiation and then receive the reflected radiation. By measuring the length of time it takes the reflected 
pulse to arrive (a technique known as time-of- flight measurement), the target’s distance can be determined by the 
equation 

d = c—. ( 1 ) 

2 

where d is the distance to the target, c is the speed of light, and t is the pulse’s round-trip time. Laser RBFs have a 
number of advantages over traditional microwave radar. Laser light has a shorter wavelength than microwaves, giving 
laser range-and-bearing finders a higher resolution and accuracy than microwave radar. Their beam width can be 
extremely narrow, giving higher angular resolution and smaller sidelobes. 

The ranges that can be achieved with a laser RBF are governed by the laser radar range equation 7 , which specifies the 
amount of light a target returns to the receiver. It is given by 


KTD\r, r 
16 7r<p 2 r A 


(2) 


where K is the profile of the transmitted beam. T is the target’s laser cross section, D is the receiver aperture's diameter, 
rj, is the transmitter’s optical efficiency, T) r is the receiver’s optical efficiency, (p is the beam width in radians, r is the 
range from transmitter to target, and P s is the transmitter’s laser power. K accounts for the intensity pattern of the beam 
at the target’s position 8 . 

Laser range-and-bearing finders have been used in space-based missions: the Near Earth Asteroid Rendezvous (NEAR) 
Laser Rangefinder 9 mapped the surface of the asteroid Eros. There have been a few experiments where laser range-and- 
bearing finders were used to determine the distance and bearing to another craft, such as the trajectory control sensor 
(TCS), but none of those experiments have resulted in widely-used sensors. To fill this need, AOS is currently 
developing a space-based laser range-and-bearing finder under the Rendezvous and Docking Sensor Suite (RDSS) 
program. 

The laser range-and-bearing finder will have 10 a working distance of 5 km in to 10 m, and will update at a rate of 5 Hz. 
Its mass will be less than 10 kg, with a maximum envelope size of 24” length by 18” width by 12” height. Its range 
accuracy will be 1 m for ranges out to 1 km, and less than 1% of the range from 1 km to 5 km. Its bearing accuracy will 
be 0.5°. While laser range-and-bearing finders often use narrow beams, the AOS laser range-and-bearing finder will use 
a wide-angle beam, covering a 5 C field-of-view. The wide beam removes the need for moving parts such as mirrors, 
prisms, or gimbals, thus reducing the sensor’s complexity and the risk of malfunctions. 

To determine the targets' bearing, the range-and-bearing finder will employ a quadrant detector. A quad detector is a 
photodetector that has been subdivided into four separate detectors. Light that falls on the detector will produce a 
current from each detector that it illuminates. By comparing the different current levels of the left two quadrants versus 
the right two, as well as those from the top two quadrants versus the bottom two, the spot centroid can be found. The 
centroid’s position will depend on where in the transmitted beam the target is. Figure 10 shows graphically how the 
signals from a quadrant detector are combined in order to determine the centroid’s position. 



Figure 10. How the signals from the quadrant detector are combined to give bearing information. 

The summation symbol indicates the sum of the signals from all four quadrants. 

The AOS laser range-and-bearing finder is designed to work well with AVGS. It uses the same retro-reflector targets as 
the AVGS and the same communication protocol. To distinguish its signals from those of AVGS, the laser range-and- 
bearing finder transmits a stream of pulses, effectively a binary signal, and uses a correlation receiver to recognize its 
specific pulse encoding. This technique also allows multiple laser range-and-bearing finders to operate near each other, 
as long as each range-and-bearing finder uses a different binary signal. 


The combination of an AVGS with a laser range-and-bearing finder provides a more complete sensor package for 
determining distance, bearing, and attitude of spacecraft. The sensor package is a crucial component for enabling 
autonomous rendezvous and capture operations. 


6. CONCLUSION 

In order to improve the safety of manned space operations as well as to enable autonomous rendezvous and docking 
missions. Advanced Optical Systems, Inc. (AOS), Orbital Sciences Corporation (OSC), and Marshall Space Flight 
Center (MSFC) have developed the Advanced Video Guidance Sensor (AVGS) as part of the Demonstration of 
Autonomous Rendezvous Technology (DART) program. AVGS provides six-degree-of-freedom information about a 
target spacecraft by illuminating a retro-reflector target, imaging the returned light, then performing pattern recognition 
on the image. For the DART mission, it does so out to distances of 300 m. 

Now that AVGS is being readied for its maiden test flight in the latter half of 2004, the next step is to improve AVGS 
performance. There are several avenues of possible improvement, including changes to or replacement of the pattern 
recognition algorithm, alterations to AVGS hardware and software, different targets, and coupling AVGS with other 
sensors. 

The current AVGS spot recognition algorithm works by thresholding the image, thus converting the image into a binary 
one of lit and unlit pixels; grouping lit pixels that are contiguous in x into line segments; aggregating lit pixels that have 
at least one pixel's worth of overlap in y into spots; calculating the centroid of each spot; and then looking for a group of 


spots whose size and configuration match the known target configuration. The algorithm, though simple, is quite robust. 
Its greatest failings are its inability to separate two returned spots that overlap and the lessening of the centroid 
calculation’s accuracy due to holes in the returned spots. 

Overlapping spots could be detangled by modifying or replacing the current algorithm. The images could be eroded or 
opened in order to separate touching spots. Histograms of either the pixel intensities or number of lit pixels along x, y, 
and possibly a non-orthogonal axis could be created, and those histograms examined for local maxima or inflection 
points. Alternatively, the current algorithm could be replaced by a circular Hough transform, which is well-suited for 
finding circles in noisy data. 

AVGS performance could be enhanced by replacing the current lens with a zoom lens, thus simplifying the lens design, 
removing holes from the returned spots, and improving the centroid calculation. The lens’s zoom would need to be 
adjusted adaptively as the target’s range changed. 

The entire acquisition and tracking process could be made adaptive. Current AVGS software has one laser power and 
imager integration time setting for each range. Newer software could vary those settings to produce a return image that 
would not be overly saturated. 

As far as AVGS targets go, rather than being made of passive retro-reflectors, they could be active targets such as 
LEDs. This would allow creation of an AVGS unit that had no on-board lasers, reducing both power requirements and 
potentially the AVGS unit’s size, and would allow targets that are visible only when activated by a signal from a 
friendly spacecraft. 

Finally, AVGS performance could be enhanced by coupling it with other space-based sensors. AOS is currently 
developing a space-based iaser range-and-bearing findei that will be compatible with AVGS. The combination of an 
AVGS and a laser range-and-bearing finder will provide range and bearing information out to 5 km, and six-degree-of- 
freedom information out to 300 to 500 m. 
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